Memory Subsystem Design for Multithreaded Processors

Multithreading processors pose new challenges and new opportunities for cache/memory hierarchy design. Multithreading significantly alters the data reference stream seen by the memory subsystem. Multithreading also demands very different performance characteristics from the cache hierarchy than a typical (uniprocessor) CPU. This paper is specifically concerned with memory hierarchy design considerations for simultaneous multithreading (SMT) processors [7]. While multithreading processors may have created a new memory throughput crisis, their latency-tolerance also provides an opportunity. We can trade memory latency in various ways to buy increased throughput-tolerance. In this paper, we examine several ways to trade latency for throughput, as well as other mechanisms for increasing throughput-tolerance. We examine memory associativity, hierarchical cacheing, and cache block size in light of these new priorities. We also consider new cache replacement as a way to reduce the L1-L2 bus utilization and implicitly to increase the over all throughput of the system.